Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Suhas-Koheda · 2024-11-13T22:16:24Z

Closes langchain4j/langchain4j#2103

Suhas-Koheda · 2024-11-13T22:17:44Z

@langchain4j hey can you please have a look at this

langchain4j

@Suhas-Koheda thanks a lot!

langchain4j-google-ai-gemini/pom.xml

langchain4j · 2024-11-14T07:45:38Z

langchain4j-google-ai-gemini/src/main/java/dev/langchain4j/googleaigemini/AutoConfig.java

+                .topP(chatModelProperties.getTopP())
+                .topK(chatModelProperties.getTopK())
+                .maxOutputTokens(chatModelProperties.getMaxOutputTokens())
+                .responseFormat(ResponseFormat.JSON)


why is it json?

Hi. Related to your question. AutoConfig it's not injecting logging properties and json format from the properties -> https://docs.langchain4j.dev/tutorials/logging/ is not working when this AutoConfig is used. I will wait until this PR is merged to create a PR to avoid conflicts or If you want to add this configuration in this PR. Whatever you consider better.
Thank you!

Hey! can you provide me with some references
As i did not find the logging used in other starters
Or maybe i overlooked
Thank you!
@franglopez

langchain4j-google-ai-gemini/src/main/java/dev/langchain4j/googleaigemini/AutoConfig.java

...ain4j-google-ai-gemini/src/main/java/dev/langchain4j/googleaigemini/ChatModelProperties.java

.../src/test/java/dev/langchain4j/googleaigemini/Langchain4jGoogleAiGeminiApplicationTests.java

langchain4j-google-ai-gemini/src/main/resources/application.properties.local

Suhas-Koheda · 2024-11-14T13:02:35Z

@langchain4j also in the timeout values i had doubt
Will it be set by the user? Or should i keep the condition if its not set then set to 0

At present i have hardcoded used 60 secs

langchain4j · 2024-11-14T13:48:31Z

@Suhas-Koheda you can see how timeouts are handled in OpenAI starter here, the same for .logRequests(chatModelProperties.logRequests()).

ddobrin · 2024-11-14T16:19:09Z

@Suhas-Koheda
when you are looking at logging requests and responses, please be aware that the GogleAiGeminiChatModel in the module langchain4j-google-ai-gemini does not make a distinction in the constructor between logRequests() and log Responses() and uses a single Boolean to enable/disable the logging functionality link

Suhas-Koheda · 2024-11-14T16:29:48Z

Yeah sure I'll take care of it!
Thank you!
@ddobrin

Suhas-Koheda · 2024-11-15T21:03:47Z

@ddobrin hey for the response format i have kept the type of prop as ResponseFormat only which is user defined - not a primitive type?
is this ok ? or does it cause any problem?

…gaugeModel

…urces/application.properties

Suhas-Koheda · 2024-11-15T23:32:37Z

@langchain4j
Hey! Hi!
I've completed writing tests and both of the tests passed!
Also review the code once and let me know if any changes are to be made :)

ddobrin · 2024-11-16T17:43:44Z

@Suhas-Koheda on a 📱 let me come back to it at the beginning of the week

ddobrin · 2024-11-30T04:06:41Z

If you like to add it and send a PR, that would be great

Suhas-Koheda · 2024-11-30T20:23:20Z

@ddobrin hey i have opened a pr for the above in the main repo and also i have chnaged the tests here also to support the same !
langchain4j/langchain4j#2217

ddobrin · 2024-12-02T20:31:04Z

Hi @Suhas-Koheda
I noticed that #2217 has been merged.

Right now, your tests are failing due to Bean instantiation issues.
If you like, you can change L66-69 to

                        "langchain4j.google-ai-gemini.streamingChatModel.safetySetting.gemini-harm-category=HARM_CATEGORY_SEXUALLY_EXPLICIT",
                        "langchain4j.google-ai-gemini.streamingChatModel.safetySetting.gemini-harm-block-threshold=HARM_BLOCK_THRESHOLD_UNSPECIFIED",
                        "langchain4j.google-ai-gemini.streamingChatModel.functionCallingConfig.gemini-mode=ANY",
                        "langchain4j.google-ai-gemini.streamingChatModel.functionCallingConfig.allowed-function-names=allowCodeExecution,includeCodeExecutionOutput"

and your tests would succeed.
It is a change from langchain4j.google-ai-gemini.chatModel to langchain4j.google-ai-gemini.streamingChatModel

Suhas-Koheda · 2024-12-02T20:32:28Z

Yeah okay!
I'll look into it tomorrow mrng! :-)
Thank you!

Suhas-Koheda · 2024-12-03T17:09:05Z

@ddobrin

completed! 🎉

ddobrin · 2024-12-03T21:02:13Z

All good @Suhas-Koheda
I can run the tests as well

cc/ @langchain4j

langchain4j

@Suhas-Koheda thank you!

@ddobrin could you please take a look at my comments as well?

...mini-spring-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/AutoConfig.java

...ng-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/ChatModelProperties.java

...ot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/EmbeddingModelProperties.java

...starter/src/main/java/dev/langchain4j/googleaigemini/spring/GeminiFunctionCallingConfig.java

...ng-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/GeminiSafetySetting.java

Suhas-Koheda · 2024-12-07T19:36:48Z

@langchain4j done!

ddobrin · 2024-12-07T20:32:12Z

@langchain4j done!

@Suhas-Koheda
Could you please replace

  @Bean
    @ConditionalOnProperty(name = PREFIX + ".chatModel.enabled", havingValue = "true")

with

@ConditionalOnProperty(name = PREFIX + ".chatModel.api-key")
@ConditionalOnProperty(name = PREFIX + ".chatModel.model-name")

Thanks

Suhas-Koheda · 2024-12-09T16:20:03Z

@ddobrin hey i have completed it!
out of curiosity, the above changes allow us defining beans with different models and such sort right?
Is that the motive?
Thank you!

ddobrin · 2024-12-09T16:48:55Z

Hi @Suhas-Koheda
this last change you have made will help us specify the fact that "auto config is set conditional on values a developer needs to set in order to establish a chat model object and interact with the LLM: the API Key and the model name"

The change for "supporting multiple models in the same config" is separate but it's good that it surfaced in the context of AutoConfig.

Suhas-Koheda · 2024-12-09T16:50:42Z

Yeah!
Now i get it!
Thank you!

langchain4j

@Suhas-Koheda thanks a lot!🙏
I've applied a few cosmetic changes, I hope you don't mind.

I've also noticed that when safetySettings or toolConfig is not specified explicitly in the properties, the whole thing fails with NPE, could you please fix this? We can merge after this is fixed. This PR will be a great addition to the next relesase which is planned for this week (on Thursday)!

langchain4j · 2024-12-16T14:01:24Z

...mini-spring-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/AutoConfig.java

+                .responseFormat(chatModelProperties.responseFormat())
+                .logRequestsAndResponses(chatModelProperties.logRequestsAndResponses())
+                .safetySettings(Map.of(
+                        // TODO NPE?


please rewrite it to avoid NPE

@langchain4j hey how can we avoid NPE like i am writing a function that checks if it is empty or not
if it is not empty the map is returned or else i wanted to return a defautl value
but what can the default value be
@ddobrin if you can help me with this :-)
Thank you

langchain4j · 2024-12-16T14:01:38Z

...mini-spring-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/AutoConfig.java

+                        chatModelProperties.safetySetting().geminiHarmBlockThreshold()
+                ))
+                .toolConfig(
+                        // TODO NPE?


Same here, please check all other model beans as well

langchain4j · 2024-12-20T11:19:10Z

@Suhas-Koheda I would love to include your PR in this release (today), but there are still some issues left (please see my comments above). Since this is an independent module, we can release it a bit later separately, once this PR is ready.

Suhas-Koheda · 2024-12-21T08:01:38Z

@langchain4j
Yeah yeha sure!
I'll be working on it
I was a bit busy with my exams
Now I'll be completely on it!

Suhas-Koheda · 2024-12-21T13:22:40Z

@langchain4j hey i have written the NPE checks can you please have a look over this
Thank you!

langchain4j · 2024-12-22T07:21:46Z

...mini-spring-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/AutoConfig.java

+            return defaultMap;
+        }
+        return Map.of(
+                safetySetting.geminiHarmCategory(),


Are these settings supposed to have only a single key-value pair?

The safety settings builder takes the map of geminiharmcategory as key and geminiharmblockthreshold as value

No it can take any number of values
It changes them into list

But for a single model thre would be only one harmcategory and blockthreshold right?

If that's not the case we can write such that the harmcategory and blockthreshold takes comma separated values and we manually convert then into map in the autoconfig?

@langchain4j

private Map<GeminiHarmCategory,GeminiHarmBlockThreshold> checkSafetySettingForNull(GeminiSafetySetting safetySetting) { if(safetySetting==null){ Map<GeminiHarmCategory,GeminiHarmBlockThreshold> defaultMap= new HashMap<>(); defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED); return defaultMap; } Map<GeminiHarmCategory,GeminiHarmBlockThreshold> userMap= new HashMap<>(); safetySetting.geminiHarmCategory().forEach(category -> userMap.put(category,safetySetting.geminiHarmBlockThreshold().get(safetySetting.geminiHarmCategory().indexOf(category)))); return userMap; }

this can be done if there are multiple GeminiHarmCategory and GeminiHarmBlockThreshold

and the test cases run properly too

the code is ready to be commited
thank you!

Hi @ddobrin, could you please help with the review?

It seems to me that current logic in checkSafetySettingForNull is not correct. If settings are not specified, defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED); is set (is it correct?). Also, chatModelProperties can have only a single setting...

@Suhas-Koheda I guess ChatModelProperties should have a Map<GeminiHarmCategory, GeminiHarmBlockThreshold> safetySettings instead of GeminiSafetySetting safetySetting.

Or List<GeminiSafetySetting> safetySettings, the same way as it is in the BaseGeminiChatModel

@Suhas-Koheda I guess ChatModelProperties should have a Map<GeminiHarmCategory, GeminiHarmBlockThreshold> safetySettings instead of GeminiSafetySetting safetySetting.

Yeah this can be done so that while setting in the properties file it will be in a more understanable way
yeah ill change it!

Hi @Suhas-Koheda

The safety filters settings, with their defaults, are available at his link

For flash and pro 1.5 the "default" block method is "SEVERITY" with the threshold by default at "BLOCK_MEDIUM_AND_ABOVE".

The HARM_CATEGORY_CIVIC_INTEGRITY filter is off by default.

Can I suggest that you do not set a value at all in
private GeminiMode checkGeminiModeForNull(GeminiFunctionCallingConfig geminiFunctionCallingConfig)?

To simplify, you can use the GeminiSafetySetting class which has a pair of <category, threshold?

Suhas-Koheda · 2024-12-24T04:39:56Z

@langchain4j hey! :-)

Suhas-Koheda · 2025-01-03T19:29:25Z

@langchain4j this logic looks perfect to me now!
jus that default values have to be sorted

ddobrin · 2025-01-04T01:03:26Z

Let me have a look tomorrow

Suhas-Koheda added 9 commits November 14, 2024 02:23

Base Commit for Google Gemini AI Model

89fa967

Added Configuration files for Google Gemini AI - ChatModel

cb8544f

Added Configuration files for Google Gemini AI - ChatModel

3a8765c

Check

0dfbc0d

Adding Streaming Google AI Chat Model and Removing Lombok uses

8335e67

Adding application.properties.local for easy usage

84bc5e3

Ignore all the .idea folders in subfolders

6d7c4cf

Ignore all the .idea folders in subfolders

300cfa2

Ignore all the .idea folders in subfolders

9618410

langchain4j reviewed Nov 14, 2024

View reviewed changes

Changing pom.xml headers

7ee36c6

Suhas-Koheda force-pushed the main branch from 8581056 to 7ee36c6 Compare November 14, 2024 11:46

Suhas-Koheda added 3 commits November 14, 2024 17:19

Resolving API_KEY to apikey

f696e82

Resolving API_KEY to apikey

4b8f2ec

Resolving API_KEY to apikey

9453ced

Suhas-Koheda added 2 commits November 16, 2024 02:28

Change Response Format to User Defined Props

4f267ff

Change Response Format to User Defined Props

affefe3

Suhas-Koheda added 5 commits November 16, 2024 03:59

Write test to check working of ChatLangaugeModel and StreamingCHatLan…

76053d9

…gaugeModel

Complete Test for Chat Language Model

6ee971e

Complete Test for Chat Language Model

637cd0e

Implement test for Streaming Chat Language Model

3e5fca5

Delete langchain4j-google-ai-gemini-spring-boot-starter/src/main/reso…

cfc5cae

…urces/application.properties

Changing the safteySetting and toolConfig and writing test

7554b25

Merge branch 'langchain4j:main' into main

4afe606

ddobrin mentioned this pull request Dec 3, 2024

Google AI Gemini Spring Boot Starter #92

Open

8 tasks

Resolving silly mistake

69660a4

langchain4j reviewed Dec 5, 2024

View reviewed changes

Resolving issues

3107cb9

Changing bean properties

56d7546

cosmetics

ed5afe0

langchain4j reviewed Dec 16, 2024

View reviewed changes

Suhas-Koheda added 2 commits December 21, 2024 18:18

Merge branch 'langchain4j:main' into main

9b52b6e

Adding NPE check for Safety Setting and Tool Config

9c167d1

langchain4j reviewed Dec 22, 2024

View reviewed changes

Changing safetySettings to map of harm category and harm threshold

816652c

Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Are you sure you want to change the base?

Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Conversation

Suhas-Koheda commented Nov 13, 2024 • edited by langchain4j Loading

Suhas-Koheda commented Nov 13, 2024

langchain4j left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

franglopez Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Suhas-Koheda Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Suhas-Koheda commented Nov 14, 2024

langchain4j commented Nov 14, 2024

ddobrin commented Nov 14, 2024 • edited Loading

Suhas-Koheda commented Nov 14, 2024 • edited Loading

Suhas-Koheda commented Nov 15, 2024

Suhas-Koheda commented Nov 15, 2024

ddobrin commented Nov 16, 2024

ddobrin commented Nov 30, 2024

Suhas-Koheda commented Nov 30, 2024

ddobrin commented Dec 2, 2024 • edited Loading

Suhas-Koheda commented Dec 2, 2024

Suhas-Koheda commented Dec 3, 2024

ddobrin commented Dec 3, 2024

langchain4j left a comment

Choose a reason for hiding this comment

Suhas-Koheda commented Dec 7, 2024

ddobrin commented Dec 7, 2024

Suhas-Koheda commented Dec 9, 2024

ddobrin commented Dec 9, 2024

Suhas-Koheda commented Dec 9, 2024

langchain4j left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

langchain4j commented Dec 20, 2024

Suhas-Koheda commented Dec 21, 2024

Suhas-Koheda commented Dec 21, 2024

Choose a reason for hiding this comment

Suhas-Koheda Dec 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Suhas-Koheda commented Dec 24, 2024

Suhas-Koheda commented Jan 3, 2025

ddobrin commented Jan 4, 2025

Suhas-Koheda commented Nov 13, 2024 •

edited by langchain4j

Loading

franglopez Nov 14, 2024 •

edited

Loading

Suhas-Koheda Nov 14, 2024 •

edited

Loading

ddobrin commented Nov 14, 2024 •

edited

Loading

Suhas-Koheda commented Nov 14, 2024 •

edited

Loading

ddobrin commented Dec 2, 2024 •

edited

Loading

Suhas-Koheda Dec 22, 2024 •

edited

Loading